1. Field of the Invention
This invention pertains generally to remote software application testing. More particularly, the invention is a method for remotely testing application program interface function calls within a software application under test which does not require a priori programming of remote procedure calls or other function-specific algorithms within the software application under test. The invention provides a lookup agent within the software application under test for determining indirect reference values and a component test framework for handling messaging between the software application under test and the test application.
2. The Prior Art
It is often necessary to test a software application in a configuration where the test program or application resides in or is executing in a different computer from the software application under test. In other cases, the test application resides on the same computer as the software application under test, but the test application executes within a different memory address space from the software application under test. In either case, the test application is considered xe2x80x9cremotexe2x80x9d from the software application under test, because the test application does not have access to the memory address spaces used by the software application under test. Thus the test application is not able to determine certain values ascertainable from the software application under test.
For example, the test application is not able to determine values which are xe2x80x9cindirectly referencedxe2x80x9d by the software application under test. Values such as structures and arrays or values containing pointers to other structures, arrays or values, for example, are indirect references having values which are not normally determinable from the test application, because the address space reference by such indirect references are not available to the test application as is well known in the art.
This inability to determine the values of indirectly referenced objects presents a serious problem in the field of remote testing because in order for the remote test application to check values of arguments in functions calls, including application program interface (API) function calls, made by the software application under test, the values of such arguments within the function calls must be also determinable. However, since function calls may include one or more arguments which include indirect references, the values of which are inaccessible remotely as described above, remote testing of such function calls involving indirect references is severely limited accordingly.
The current approach to providing the actual values of indirect references provided in function calls involves xe2x80x9cpre-packagingxe2x80x9d function calls. Where function calls include arguments having indirect references, this approach extracts the values of such arguments a priori from memory and/or using a local call to obtain such indirect reference values. After determining the value of each indirect reference, the function call including the actual values packaged therewith is suitable for use by the remote test application.
Several disadvantages are presented by the prior art approach of pre-packaging. First, the task of pre-packaging function calls involves a substantial amount of development time in tracking the actual value of interest, particularly because indirect references may include further indirect references. For example, a first pointer object may reference a structure object which further references a second pointer object. Thus, ascertaining the actual value of an indirect reference may involve tracking a plurality of other indirect references. Second, as structures or function calls are changed, the pre-packaging effort of tracking the actual values of indirect references must be repeated to ensure accurate manual referencing in the software application under test resulting in additional development time and costs.
These disadvantages are also present with systems using remote procedure call (RPC). RPC is a type of protocol that allows a client program on one computer to execute a program on a server computer. Using RPC, a system developer need not develop specific procedures for the server. Instead, the client program sends a message to the server with appropriate arguments and the server returns a message containing the results of the program executed. The arguments provided by the client messages to the server must be resolved to actual values if the arguments involve indirect references in order to provide remote testing of such messages, because as noted above, remote test applications do not have access to the memory address space pointed to by indirect references. The process of resolving indirect references to actual values requires a development effort and expense similar to that described above for pre-packaging function calls.
Furthermore, it is often disadvantageous to have the developer of the software application under test provide the code required to make such software application xe2x80x9ctest-readyxe2x80x9d (i.e., providing code to make such application suitable for test) because the developer may be unfamiliar with the techniques used by a tester to test software applications. Likewise, it is disadvantageous to have the tester provide code within the software application under test to make such application xe2x80x9ctest-readyxe2x80x9d because the tester may be unfamiliar with the specifics of the software application code. It is desirable to have the software application under test be independent from the test application wherein the tester need only be familiar with the function calls provided by the software application under, not the specific implementation of the function calls.
Accordingly, there is a need for a method and system for remote testing of software applications which does not require a priori programming of remote procedure calls or other function-specific algorithms within the software application under test. The present invention satisfies these needs, as well as others, and generally overcomes the deficiencies found in the background art.
An object of the invention is to provide a method and system for remote testing of software applications which overcomes the prior art.
Another object of the invention is to provide a method and system for remote testing of software applications which does not require a priori programming of remote procedure calls or other function-specific algorithms within the software application under test.
Another object of the invention is to provide a method and system for remote testing of software applications which provides a lookup agent within the software application under test for determining indirect reference values and a component test framework for handling messaging between the software application under test and the test application.
Another object of the invention is to provide a method and system for remote testing of software applications which provides a method for ascertaining the values of indirect reference objects without function-specific algorithms within the software application under test.
Another object of the invention is to provide a method and system for remote testing of software applications having function calls with one or more arguments including an indirect reference which does not require function-specific algorithms within the software application under test.
The present invention is a system and method for remotely testing function calls made by a software application by providing a mechanism for externally intercepting the function calls made by the software application and a mechanism for determining the values of arguments having indirect references within each of the function calls.
In its most general terms, the system of the invention provides a lookup agent, a component test framework (CTF) operatively coupled to the software application under test (component under test), and a test application operatively coupled to the CTF.
The lookup agent carries out the operation of ascertaining indirect reference values contained within the component under test. The software code of the lookup agent is provided or included within the code of the component under test and operates in the same address space as the component under test to thereby enable the lookup agent to have access to the memory address space of the component under test. Under this arrangement, the lookup agent is able to ascertain the values of structures, arrays, pointers and other data types having indirect references.
The component test framework (CTF) carries out the operation of handling messaging between the component under test and the test application. Normally, the CTF operates in a different memory address space from the component under test but is configured to ascertain or otherwise intercept function calls made by the component under test. For example, the component under test may include one or more function calls to another component or a remote program (other component). During interception of these function calls, the CTF also ascertains the arguments provided in the function calls. As noted above, some of these arguments may be actual values while other arguments may include indirect references. The CTF transmits the function calls including its associated first-level arguments to the test application for testing. As described further below, the test application includes an algorithm for testing each of the function calls and providing a resultant value to each of the function calls. The CTF then provides this resultant value to the component under test as a response to each of the function calls made by the component under test.
The CTF also manages commands issued by the test application to the lookup agent and results provided by the lookup agent to the test application. For example, the test application may request the value of a particular argument which contains an indirect reference. The CTF communicates this request from the test application to the lookup agent, which in response to the request, accesses the memory address of the argument requested to ascertain the value of the argument, and then provides the argument value to the CTF. The CTF receives the argument value and communicates the value to the test application for further processing. In certain cases the argument value returned to the test application is another indirect reference, in which case the above described steps for requesting the value of the argument is repeated for the new argument value returned.
The test application carries out the operation of testing, among other things, the function calls made by the component under test. As noted above, each function may include one or more arguments, some of which may be actual values while other arguments may include indirect references. In order for the test application to check the values of the arguments to a function call, the test application must have access to the argument values.
During testing, the CTF will intercept and transmit function calls made by the component under test to the test application for testing. The test application tests or checks each function call transmitted from the CTF using a variety of means as is known in the art. For example, the CTF may intercept an xe2x80x9copenxe2x80x9d function call. Any arguments used in the function call that include an indirect reference must be resolved to the actual data value referenced by the arguments.
To obtain the argument""s value, the test application makes a request to the lookup agent via the CTF, and the value of such an argument is returned from the lookup agent to the test application via the CTF as described above. However, in some instances, the argument value returned to the test application is another indirect reference, in which case the above described steps for requesting the value of the argument is repeated again for the argument value returned.
Once all the indirectly referenced arguments are resolved to their actual data values, a check of the function call can then be made using means well known in the art. In an illustrative body of the invention, the test application may directly reply to a function call by providing a resultant value for the function call which is communicated to the component under test via the CTF. Alternatively, the test application may simply xe2x80x9cauthorizexe2x80x9d the function call in which case the function call is carried out directly with the xe2x80x9cother componentxe2x80x9d which provides the resultant value of the function call to the component under test. Where errors are detected in the function call, the test application may provide a message to the test user of the invention to identify the error detected. Other error-checking mechanisms known in the art may also be used with the invention.
Further objects and advantages of the invention will be brought out in the following portions of the specification, wherein the detailed description is for the purpose of fully disclosing the preferred embodiment of the invention without placing limitations thereon.